After I released Test-Run version 0.0111, and made some related Test-Run-CmdLine releases, I noticed that Test-Run-CmdLine got broken. As it turned out, a regression was introduced into Test-Run which, due to inadequate testing in the core distribution, was only discovered in the Test-Run-CmdLine tests. I could have discovered it on my machine, if only I had ran my script to build all the Test-Run-related distributions.
Namely, in my continuous work on converting the legacy GPL and (original) Artistic code that I inherited from Test::Harness to an MIT-X11 Licensed code, while refactoring it in the process, I broke the "leaked directory" (which indicates which files inside it were leaked). After I added a regression test for it in the core Test-Run distribution, I found out that I had about three bugs in the new code, which was about 20 lines. (Reminds of "How to fit 3 bugs in 512 bytes of security code"). After I fixed them, Test-Run-CmdLine did not complain.
There are several lessons here. One is that one should also test modules that are based on the core module before releasing it. The other is that if bugs are discovered by testing a derived module, then regression tests should also be added to the test of the core module. And finally, that it is a good idea to have a good test coverage before refactoring.
